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ABSTRACT 

The primary purpose of this document is to provide 
accrediting organizations with guidance about how to engage evidence of 
student learning as they establish policies, standards, and approaches to 
review. As a result, it addresses three main topics: (1) accreditors must be 

clear about terminology when considering evidence of student learning 
outcomes; (2) accreditors have to face a number of policy choices about 
evidence of student learning outcomes; and (3) they will encounter a similar 
set of issues when examining student learning that need to be identified 
systematically. Accrediting organizations must be more aggressive and 
creative in requiring evidence of student learning outcomes as an integral 
part of their standards and processes for review. The accrediting community 
must become more vocal and articulate in talking about evidence of student 
learning outcomes, and each individual accreditor must choose a coherent 
rationale that underlies its approach to student learning outcomes, and then 
must use this rationale to explain the procedures it adopts . Accreditors need 
a language with which to talk with one another about what they are doing. All 
accreditors would benefit form the development of common resources. The 
growing body of accreditation experience should be organized for use by the 
community in common. (SLD) 
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“First, the task of addressing student learning outcomes has no single or simple answers. 

Second, it is imperative that accreditors take it on. What is required for them to do so? 

Three lines of collective action seem warranted. 

■ First, the accrediting community needs to develop a coherent and understandable 
way to explain its collective approach to the matter of evidence of student learning 
outcomes to outside stakeholders. This rationale must make it clear that there are 
appropriate differences among accreditors in how they choose to engage student 
learning, but that all are doing so in appropriate and rigorous ways. But it must also 
make clear that they are requiring some measure of direct student attainment — not 
merely proxies for it — in their various approaches. 

■ Second, accreditors need a language with which to talk to one another about what 
they are doing. This may or may not require the development of a common vo- 
cabulary. But it does require a common conceptual framework that allows varied 
members of the community to understand the key distinctions and similarities 
among their approaches. 

■ Third, all would benefit from the development of common resources . These might 
include sharing review approaches and techniques like standards of evidence or 
ways to “audit” institutional or program assessment efforts. It might also include 
more fundamental research and development efforts directed toward creating better 
tools for examining learning or identifying best practices. There is a growing fund 
of experience to build on here, both within and outside the accreditation commu- 
nity. But the growing body of experience has not yet been effectively “rounded up” 
for use by the community in common.” 



From Concluding Thoughts y page 24 
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Accreditation and Student Learning Outcomes 
A Proposed Point of Departure 



Introduction 

Student learning outcomes are rapidly taking center stage as the principal gauge of 
higher educations effectiveness. Employers and elected officials have never been clearer 
in their demand that the graduates of Americas colleges and universities should possess 
an increasingly specific set of higher order literacies and communications skills. Parents 
and the public — acting as “consumers” of higher education — are looking not just at 
price, but at the underlying quality of a college credential and what it will buy them in 
the employment marketplace. Inside the academy (though admittedly not as forcefully) 
conversations are widening about how to better “organize” institutions of higher edu- 
cation for improved learning. Meanwhile, the growing presence of technology and 
distance delivery in the landscape of higher education by its very nature enhances the 
centrality of student learning outcomes. Absent traditional markers of achievement 
like numbers of classes completed or credits earned, alternative instructional approaches 
can only demonstrate their relative effectiveness in terms of what students have learned. 

Accreditation organizations have responded to the growing salience of learning 
outcomes in a variety of ways. Virtually all now include explicit references to student 
learning in their standards for accreditation. Most also require institutions or programs 
to examine student achievement or “institutional effectiveness” as part of their self- 
study and review processes — usually in the form of some kind of “assessment.” In fact, 
attention to this issue on the part of accreditors in some cases dates back to the mid- 
1980s. But after more than a decade, the question of exactly bow accreditation should 
engage the topic remains unclear. In particular, what is meant by “assessment” often 
varies greatly — embracing everything from job placement, through student satisfaction, 
to self-reported gains in skill and knowledge on the part of students and former stu- 
dents. None of these constitute direct evidence of student learning outcomes of the 
kind currently being asked for by external stakeholders. 

How should accrediting organizations best position themselves to meet these 
conditions? On the one hand, it is clear that accreditation must respond effectively 
and coherently if it is to maintain its central role in quality assurance. On the other, 
accrediting organizations legitimately differ in the particular postures and levels of 
engagement that they will adopt with respect to this issue. The primary purpose of this 
document is to provide accrediting organizations with guidance about how to engage 
evidence of student learning as they establish policies, standards, and approaches to 
review. As a result, it addresses three main topics. First, accreditors must be clear about 
terminology and should speak with a common voice when considering evidence of 
student learning outcomes. Second, accrediting organizations have to face a number 
of policy choices as they approach the matter of evidence of student learning, and it 
is important to know something about the nature of these choices and their conse- 
quences. Finally, all accreditors will encounter a similar set of issues when they examine 



0 



'CREDITATION AND STUDENT LEARNING OUTCOMES: A PROPOSED POINT OF DEPARTURE 



All Accreditors: 

■ Must be clear 
about terminology 
when considering 
evidence of student 
learning outcomes; 

■ Have to face a 
number of policy 
choices about evi- 
dence of student 
learning outcomes; 
and 

■ Will encounter a 
similar set of issues 
when examining stu- 
dent learning that 
need to be system- 
atically identified. 
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student learning, so systematically identifying these issues and discussing how they 
might appropriately be addressed is a useful exercise. 

Throughout the document, we hope two things are clear. First, accreditation must 
adopt a visible and proactive stance with respect to assuring acceptable levels of student 



academic achievement. Given escalating stakeholder demands, nothing less will prove 

sufficient to maintain the public credibility of our voluntary 
peer-based system. But just as important, we believe it is 
imperative to avoid orthodoxy. Accrediting organizations 
can and should differ in the ways in which they approach 
the specific tasks associated with assuring quality — though 
the reasons for the choices they make must be understand- 
able and defensible. Both commitments are important if 
we are to make progress as a community. 



■ Accrediting organizations — both institu- 
tional and specialized — must become more 
aggressive and creative in requiring evidence 
of student learning outcomes as an integral 
part of their standards and processes for 
review. 



■ The accrediting community as a whole 
must become more vocal and articulate in 
talking about evidence of student learning 
outcomes, and must evolve a common 
language with which to do so. 

■ Each individual accreditor must deliberately 
choose a coherent rationale that underlies its 
own approach to student learning outcomes, 
and must be able to use this rationale to 
dearly and publicly explain the specific 
procedures that it adopts. 

that was highly literate, well-versed in problem solving 
and collaborative skills, and equipped with appropriate technical skills to meet the 
needs of an emerging “knowledge economy.” Growing uneasiness within the academy, 
meanwhile, centered on loss of coherence in the undergraduate experience; at the same 
time, academic leaders were aware of the evident potential of research-based knowledge 
about how to occasion better learning among college students (e.g., Involvement in 
Learning, 1984; Integrity in the College Curriculum , 1985). Together, these stimuli 
helped spawn a recognizable “assessment movement” in higher education, evidenced 
by an expanding array of conferences, publications, measurement techniques, and 
institutional experiments. 

Some accrediting organizations were pioneers in this movement. Among regional 
accreditors, the Southern Association of Colleges and Schools (SACS) adopted an 
“institutional effectiveness” standard in 1986 that required every institution under 
review to provide explicit evidence about its attainment of established goals — among 
them, goals for student learning. Taking a somewhat different tack, the North Central 
Association (NCA) required all member institutions to prepare “assessment plans” 
that focused directly on evidence of “student academic achievement.” At the same 
time, NCA developed workshops and materials designed to help institutions with the 
task of assessment and began training its peer reviewers to evaluate institutional efforts 
to examine student learning outcomes. Other regionals followed similar paths — 
developing handbooks and workshop materials to support institutional efforts and 



What Is the Problem and How 
Has Accreditation Responded? 

Systematic public concern about what and how much 
students are learning in American colleges and universities 
dates back to the mid-1980s. One stimulus for this concern 
began in K-12 education, which was felt to be under- 
performing at “crisis” levels (e.g., A Nation at Risk, 1983), 
and moved up the “grade ladder” into public postsecondary 
settings (e.g., Time for Results, 1986). Another arose simulta- 
neously among employers and business leaders increasingly 
focused on the need to develop a “21st Century” workforce 
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gradually raising the salience of “assessment” in their respective self-study and review 
approaches. By the mid-1990s, all had some kind of policy in place. 

Selected specialized accrediting organizations were also early proponents of assess- 
ment. In the late 1980s, the American Association of Collegiate Schools of Business 
(AACSB) experimented actively with a direct assessment of core business subjects for 
adoption by its members on a voluntary basis.* The Accreditation Board of Engineering 
Technologies (ABET), meanwhile, adopted a new outcomes-based approach to accred- 
iting engineering programs (ABET 2000) that centered on evidence of student learning 
in key engineering disciplines. The various health professions — always outcomes- 
focused because of professional licensing requirements — also moved strongly toward 
evidence of student attainment as a major component of their recognition processes. 

While motivated in part by internal and professional concerns, these early actions 
by accreditors were influenced by an increasingly active stance by the U.S. Department 
of Education (DOE). Indeed, in 1989, federal regulations first required accrediting 
organizations to examine student learning outcomes as a condition of recognition. 

Since that time, these requirements have become more explicit and stringent. Indeed, 
the latest criteria governing DOE review of accrediting organizations prominently notes 
measurement of “student academic achievement” as a condition of recognition. 

State governments were also prominent players in the early assessment movement — 
far more prominent, at first, than accrediting organizations. For better or worse, states 
like Florida, Tennessee, and Georgia already had comprehensive general examinations 
in place for students attending public institutions, and by the late 1980s they were 
joined by others such as New Jersey and South Dakota. Other states including Virginia, 
Colorado, and South Carolina adopted “institution-centered” assessment approaches 
that allowed institutions to select their own assessment methods consistent with estab- 
lished goals for student learning, while requiring them to report periodically and 
publicly on results. By 1989, about half of the states had a requirement of this latter 
kind in place. As regional accrediting organizations gradually moved into assessment 
in the 1990s, though, many states appeared happy to allow them to assume the burden 
of reviewing institutional assessment programs. Wisconsin and North Dakota, for 
example, explicitly referenced NCA assessment requirements as part of their own 
accountability processes, while several southern states adopted SACS criteria as part 
of their own “report cards” or “performance indicators.” 

A range of other forces have stimulated — indeed, made imperative — accreditations 
interest in examining student learning outcomes. Foremost among them are rapidly 
changing modes of instructional delivery and a burgeoning competency movement in 
corporate training. At first, distance education was of concern to accreditors because it 
required institutions to demonstrate essential equivalence in quality between on-campus 
instructional programs and instruction delivered at a distance. But in order to do this 
effectively, some way to judge the comparability of learning outcomes was required. More 
subdy, as distance delivery became asynchronous and student-centered, demonstrated 
student mastery of the subject matter became the only way in which academic progress 
could be judged. The result has been increasing pressure on accreditors to develop review 
approaches that are capable of looking at instructional programs that are not anchored in 



"AACSB changed its name to AACSB International — The Association to Advance Collegiate Schools of Business in 200 1 . 
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“seat time,” and that involve resource levels and configurations quite different from those 
which their established standards were designed to address. The competency movement, 
visible in a rapidly growing array of achievement-based credentials in professional and 
technical fields, is exerting similar pressures. Some postsecondary institutions and pro- 
grams are adopting such approaches themselves to certify achievement — especially in 
technical and professional fields. And there is growing pressure by students and profes- 
sional associations to find ways to recognize such certificates in the form of transfer 
equivalencies, so that students can continue their education at established colleges and 
universities. Both these developments have important implications for accrediting 
organizations and are forcing them to pay ever-greater attention to learning outcomes. 

Accrediting organizations have thus not been idle in the face of escalating needs 
to demonstrate what college students know and can do. But they have responded 
in quite different ways and have moved at different speeds to implement new 
approaches. Furthermore, evidence is strong that institutions and programs remain 
only marginally engaged. Few have progressed beyond superficial engagement with 
“assessment,” though accrediting organizations have been asking them to do so for 
years. Meanwhile, the demands for accountability and the changes in instructional 
delivery that originally stimulated national concern about student learning outcomes 
are unabated. At minimum, these conditions suggest: 

■ Accrediting organizations — both institutional and specialized — must become more 
aggressive and creative in requiring evidence of student learning outcomes as an 
integral part of their standards and processes for review. 

■ The accrediting community as a whole must become more vocal and articulate in 
talking about evidence of student learning outcomes, and must evolve a common 
language with which to do so. 

■ Each individual accreditor must deliberately choose a coherent rationale that 
underlies its own approach to student learning outcomes, and must be able to 
use this rationale to clearly and publicly explain the specific procedures that it 
adopts. 

While the specific means that accreditors select will legitimately differ, not address- 
ing these conditions is no longer an option. At the same time, speaking with a common 
voice as a community cannot help but reinforce the position of peer-based quality 
assurance in uncertain times. 

Defining the Territory 

While accreditation has always been concerned about student academic achievement, 
individual accrediting organizations have varied widely in both their levels of engage- 
ment and in the particular ways they have approached the topic. They have also con- 
ceptually “mapped the territory” in different ways, and have employed a bewildering 
array of terms to describe what they do. Partly this is the result of independent develop- 
ment. While familiar elements of the academic landscape like “courses,” “curricula,” 
“programs,” “faculty,” and “standards of academic progress” have evolved reasonably 
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common meanings through continuing use, linguistic distinctions among labels like 
“outcomes,” “learning,” “assessment,” and “effectiveness” remain relatively underdevel- 
oped. Each commission or agency begins from its own perspective and has naturally 
concentrated on those elements of the topic that are closest to its own interests. Dis- 
cussing “outcomes” sensibly requires a conceptual approach that can appropriately 
distinguish a) different levels of analysis (e.g., institution, program, individual student), 
b) different kinds of “results” of an academic experience (e.g., cognitive learning, career 
success, or satisfaction) , and c) different perspectives or viewpoints (e.g., attainment 
levels at or after graduation, “value-added” or change while enrolled, etc.). Absent 
such an underlying conceptual scheme, any definitions would be arbitrary and hard 
to explain even if some “common” terminology were adopted. 

One way to begin to evolve such a common language is to think systematically 
about each component of the core concern: evidence of student learning outcomes. 

Doing so first requires discussion of what we mean by an “outcome” and how it is 
different from other potential dimensions of performance. Second, it demands dis- 
tinctions among units of analysis — at minimum individual students, curricula and 
academic programs, and institutions. Third, it requires us to explicitly distinguish 
“learning” from other kinds of “good effects” that students may experience as a result 
of participating in a postsecondary experience. And finally, it necessitates specific 
consideration of how we know whether (and to what degree) any of these results has 
occurred, and to what causes we can attribute them. 

■ “Outcomes” vs. “Outputs.” While an outcome is clearly a result of institutional 
and student activities and investments, not all results are properly considered 
outcomes. Numbers of graduates, numbers of credits produced through instruc- 
tion, or types of service or research products generated, for example, are clearly 
results of what an institution does. But they are most usefully defined as outputs 
of higher education. Other dimensions of institutional or program performance 
like efficiency or productivity are equally the results of what an institution does, 
and assessing them may be important for some evaluative purposes. But they are 
not the same thing as outcomes. Indeed, this is the whole idea behind the notion 
of institutional effectiveness , which examines the extent to which the institution 

as a whole attains all of the performance goals it establishes for itself. Although 
outputs and performance are predominantly institution-level terms, moreover, 
outcomes are only visible by aggregating what happens to individual students. 

An outcome , therefore, can be most broadly defined as something that happens 
to an individual student (hopefully for the better) as a result of his or her atten- 
dance at a higher education institution and/or participation in a particular 
course of study. 

■ “Learning” as a Special Kind of Outcome. Similarly, relevant and valuable 
outcomes are not confined to learning because students may benefit from their 
engagement in postsecondary educational experiences in many other ways. Addi- 
tional behavioral outcomes or expediences that may result include employment and 
increased career mobility, enhanced incomes and lifestyles, the opportunity to 
enroll for additional education, or simply a more fulfilled and reflective life. 

■ CREDITATION AND STUDENT LEARNING OUTCOMES: A PROPOSED POINT OF DEPARTURE 



Discussing “out- 
comes” sensibly 
requires a concep- 
tual approach that 
can appropriately 
distinguish a) different 
levels of analysis 
(e.g., institution, 
program, individual 
student), b) different 
kinds of “results” 
of an academic 
experience (e.g., 
cognitive learning, 
career success, or 
satisfaction), and c) 
different perspectives 
or viewpoints (e.g., 
attainment levels at 
or after graduation, 
“value-added” or 
change while 
enrolled, etc.). 
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Hopefully these are related to learning in some way. Indeed, evidence that students 
have obtained such benefits is often used by institutions as a proxy for instructional 
effectiveness. But such subsequent experiences, however successful, should not be 
confused with actual mastery of whats been taught. Although equally an outcome 
and frequently examined, student satisfaction with the college experience should 
not be confused with learning. Certainly satisfaction is important — especially as 
it is related to persistence and therefore a continued opportunity to learn. Student 
learning outcomes , then, are properly defined in terms of the particular levels of 
knowledge, skills, and abilities that a student has attained at the end (or as a result) 
of his or her engagement in a particular set of collegiate experiences. 

■ Learning as “Attainment.” Defined in terms of the levels of attainment achieved, 
however, requires learning outcomes to be described in very specific terms. While 
institutions, disciplines, and professions vary considerably in the ways (and the 
extent to which) they are described, several broad categories can usually be dis- 
tinguished. Knowledge outcomes generally refer to particular areas of disciplinary 
or professional content that students can recall, relate, and appropriately deploy. 
Skills outcomes generally refer to the learned capacity to do something — for 
example, think critically, communicate effectively, productively collaborate, or 
perform particular technical procedures — as either an end in itself or as a pre- 
requisite for further development. Attitudinal or affective outcomes , in turn, usually 
involve changes in beliefs or the development of particular values — for example, 
empathy, ethical behavior, self-respect, or respect for others. Learned abilities , 
finally, typically involve the integration of knowledge, skills, and attitudes in 
complex ways that require multiple elements of learning. Examples embrace 
leadership, teamwork, effective problem solving, and reflective practice. All such 
taxonomies require institutions or programs to define learning goals from the 
outset as guides for instruction and for judging individual student attainment. 
Expressed in terms of competencies , moreover, such goals describe not only what 

is to be learned but also the specific levels of performance that students are 
expected to master. Certification y finally, implies that these specific levels have 
actually been attained. 

■ “Learning” as Development. In many cases, institutions and programs describe 
student learning not just in terms of attainment but in terms of growth or en- 
hancement. While this construction emphasizes the unique contribution of the 
educational program to current levels of student attainment, it also requires 
some knowledge of what levels of attainment were like before the student en- 
rolled. Value-added , “before-after,” or net effects are terms that are frequently used 
to describe such longitudinal ways of looking at development. This perspective 
is, of course, not confined to student learning. For example, many instructional 
programs base their claims of effectiveness on things like enhanced income, 
changes in career, or even increased satisfaction. From the standpoint of quality 
assurance, both attainment and development may be important. Certification of 
specific levels of knowledge, skill, or ability for a given program completer — for 
example, in the form of a licensure examination — is thus intended to guarantee 
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that the certified individual is able to perform competently under a variety of 
circumstances. Evidence of this kind will be a particularly important element of 
“quality” for employers seeking to hire such individuals or the clients who seek 
their services. But evidence about value-added or net effects will be especially 
important elements of “quality” for prospective students who are looking for 
institutions and programs that will benefit them the most, or for policymakers 
and the public who seek maximum payoff for the resources that they have 
invested. In either case, it is important to be clear about definitions: student 
learning outcomes should refer normally to competencies or attainment levels 
reached by students on completion of an academic program; if development or 
value-added is intended as well, this meaning should be explicitly stated. 

■ Assessment and Learning. Assessment , finally, refers primarily to the methods that 
an institution or program employs to gather evidence of student learning. But 
historically, the term has been applied in several ways. For accreditation purposes, 
the most common meaning refers to the collection and use of aggregated data about 
student attainment to examine the degree to which program or institution-level 
learning goals are being achieved. But the term assessment is also commonly used 
to describe the processes used to certify individual students or even, in some cases, 
to award grades. Evaluation also commonly refers to evidence-gathering processes 
that are designed to examine program or institution-level effectiveness. But the 
object of evaluation usually extends beyond learning outcomes to examine a much 
wider domain of institutional performance. Differences in concept and terminology 
are also apparent when describing the informational results of assessment. Here 
terms like measurement and data are often used, implying that legitimate assessment 
should only yield quantitative results. Measurements, though, are a special kind of 
evidence , which is probably the more appropriate term for accreditation purposes. 
Evidence can embrace the results of both quantitative and qualitative approaches 
to gathering information, both of which may be useful in judging learning. At 
the same time, the term evidence suggests both the context of “making and sup- 
porting a case” and the need to engage in consistent investigations that use multiple 
sources of information in a mutually reinforcing fashion. But to count as evidence 
of student learning outcomes , the information collected and presented should go 
beyond such things as surveys, interviews, and job placements to include the actual 
examination of student work or performance. As a consequence, assessment of 
student learning outcomes is most appropriately defined for accreditation purposes 
as the processes that an institution or program uses to gather direct evidence about 
the attainment of student learning outcomes, engaged in for purposes of judging 
(and improving) overall instructional performance. 

Chart 1 (see page 8) attempts to display some of these key terms in a tabular 
format so that the relationships among them are apparent. Following the logic of 
the discussion above, columns in Chart 1 correspond to the specific definitional 
distinctions revealed by parsing the term “student learning outcomes.” Within each 
column, a variety of commonly used terms is listed, each of which is appropriate 
for certain purposes. The shaded area, for example, notes those that accreditors 
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might properly choose to use when referring to evidence of student learning out- 
comes. Definitional guidance of this kind is not intended to be rigid, and it is 
expected that accreditation organizations will continue to use similar terms with 
different inflections and nuances. But common language can both enhance mutual 
communication and reinforce accreditors as they more aggressively adopt evidence 
about student learning outcomes as the primary marker of quality. 



Chart 1 

A Taxonomy of Terms Commonly Used in Connection with the “Assessment” 
of Student Learning Outcomes 



Units of 
Analysis 


Ways of Looking 
at Performance 


Ways of Looking 
at Outcomes 


Ways to Review 
Performance 


Institution 


Efficiency 

Productivity 

Effectiveness 


Behaviors 

■ Employment 

■ Further Education 

■ Career Mobility 

■ Income 


Evaluation 


Program 


Output 




Measurement 




Productivity 


Satisfaction 


Indicator 

Assessment 


Student 


Outcome 


Learning 

■ Knowledge 

■ Skill 

■ Ability 

■ Attitude/ 
Disposition 

Attainment 

Development 


Evidence of 
Achievement 

■ Examinations 

■ Performances 

■ Student Work 



Some Dimensions of Policy Choice 

Accrediting organizations have frequently acknowledged student learning outcomes 
as an important dimension of quality — and, in many cases, have actively built or 
adopted new review standards and criteria to address it. But the particular “stances” 
that they have adopted vary widely. This is clearly appropriate. Specialized and 
institutional accreditors have always differed with respect to both the topics that they 
choose to address and in the level of emphasis that they place on particular areas of 
performance. But the choices that each has made have, at least up to now, been 
largely reactive. Accreditors know that concerns about student learning are growing 
and that providing evidence that it is occurring will be an ever-increasing attribute 
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of credible quality assurance. And even if they did not, they would be reminded 
of the topics salience routinely by the DOE and state higher education authorities. 
Much has been done as a result, but most has been accomplished at the behest of 
others. 

Partly as a result, many “assessment-related” initiatives that accrediting organiza- 
tions have embarked upon have developed piecemeal in the form of a new standard 
or criterion, or a new requirement for institutional reporting. Essentially imple- 
mented as add-ons to an existing set of standards and an established framework 
for review, their development has not always demanded more basic thinking about 
the choices that a given agency must make in order to address the issue effectively. 
This is not to say that the many assessment-related policies and requirements that 
accreditors have adopted have been ineffective. Indeed, by many accounts, a large 
share of the institutional attention to the topic of taking responsibility for student 
learning outcomes that has occurred in the last decade can be attributed directly to 
the actions of accreditors. But it highlights the fact that these initiatives have not 
been particularly systematic or intentional — the result of conscious decisions made 
in the context of a deliberate set of policy choices. Identifying these policy choices 
clearly, like defining key terms, can help accreditations stakeholders make sense of 
what might otherwise appear a bewildering array of apparently different require- 
ments. Similarly, it may assist individual accrediting organizations to explain and 
justify their own unique approaches. 

For purposes of argument, three dimensions of choice collectively define the 
“posture” that an accrediting organization can adopt when approaching the task of 
assuring the quality of student learning outcomes. As depicted in Chart 2, each of these 
dimensions describes a continuum, not a set of “either/or” choices. Individual accredi- 
tors may adopt positions at any point on each, though the dimensions themselves — 
like any “scale” — are best described in terms of their extremes. 

The dimensions are as follows: 
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■ Prescription of Outcomes. This dimension defines the extent to which an accred- 
iting organization actively specifies the particular learning outcomes that should 
result from an educational program, or leaves the choice of learning goals entirely 
up to individual institutions or programs. Most regional and institutional accredi- 
tors, for example, see their primary responsibility as ensuring that the institution 
takes responsibility for assessing and achieving the unique set of learning outcomes 
that it establishes for itself. This position is consistent with a wider philosophical 
stance that sees each institutions distinctive mission and particular instructional 
philosophy as the principal point of departure for peer review. Others adopt a 
middle ground — emphasizing the assessment of unique mission-driven learning 
outcomes, to be sure — but also naming a core set of learning outcomes that ought 
to be adopted (and examined) by all institutions in some way. These are usually 
4 general education” outcomes that include such academic abilities as college-level 
written and oral communication, higher-order thinking or critical reasoning skills, 
and quantitative or problem-solving abilities. Additional broad content domains 
are sometimes named using, for example, the traditional subject area taxonomy 
of humanities, sciences, and social sciences (and occasionally the arts). 

Specialized accreditors, in contrast, must be far more specific in the learning 
outcomes that they explicitly name and for which they demand evidence of 
attainment. At minimum, these outcomes will comprise the specialized core 
knowledge, skill, and ability domains required for effective practice, though they 
may occasionally also embrace broader sets of attitudes or knowledge/skill areas 
(like teamwork or problem solving). In many cases, however, such intended student 
learning outcomes are implied by the particular areas of coursework required of an 
accredited program, rather than being explicitly stated in outcomes terms, so course 
completion is all that is required as evidence that these outcomes are being attained. 

The choice that an accreditor makes on this dimension will thus fundamentally 
shape the kinds of review approaches that it adopts. Almost by definition, if it does 
not prescribe particular outcomes, it will be less concerned with actual levels of 
attainment. It will also tend to concentrate more on examining the adequacy of the 
evidence provided by the institution or program, and less on what this evidence 
actually says about the levels of student performance. If particular important out- 
comes are specified, in contrast, there will likely be more interest in explicidy 
determining the degree to which the institution or program “measures up” to some 
common standard of performance for its graduates. What is not at issue is that goals 
must be stated explicitly in terms of student learning outcomes in both cases. In 
order to effectively address the issue, accreditors must either clearly specify the 
student learning outcomes they require institutions to address, or must require 
institutions to do so. 

■ Unit of Analysis. This dimension addresses the extent to which the principal focus 
of interest for accreditation is placed on ensuring the attainment of identified com- 
petencies by all of the graduates of the institution or program, or upon ensuring its 
overall “effectiveness” against more general goals for student learning. The extreme in 
the first case can best be illustrated by an entirely “competency-based” institution or 
program in which every student awarded a credential must be certified through some 
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form of demonstration of mastery that involves more than simply completing a cer- 
tain number of courses. Like a licensure or certification exam in a professional field, 
such a demonstration in principal “guarantees” that all graduates possess the requisite 
competencies at the required levels of mastery. Specific standards of attainment are 
thus built direcdy into the programs design and apply to all students enrolled. 
“Quality assurance” in such a context requires an external 
reviewer to look closely at the methods and policies that 
the institution or program employs to back up this guaran- 
tee. These might include its grading methods and associ- 
ated standards of academic achievement, the policies that 
it establishes to govern student progress and the award of 
credentials, and the methods that it uses to judge student 
attainment. 

At the opposite extreme of the continuum, the pri- 
mary object of investigation is not the individual student 
but rather the institution or program as a whole. This 
option is the closest to what most accrediting organiza- 
tions (and virtually all regional accrediting organizations) 
currently do. While goals for student learning may indeed 
be established in the form of intended outcomes, these 
are not put forward as “standards” that every graduate is 
expected to attain. Rather, they are advanced as templates 
against which the overall performance of the institution 
or program can be judged — and hopefully improved. Out- 
comes statements in such cases are generally less detailed 
than typical competency statements and they rarely specify 
required levels of attainment. As a result, a variety of meth- 
odologies (such as examinations, portfolios, or surveys) are usually employed to 
generate a body of evidence designed to demonstrate the degree to which estab- 
lished goals are being met. These assessments rarely apply to all students and, rather 
than being embedded in the programs instructional and curricular designs, are 
generally “added on” to them in some way. “Quality assurance” in such a context 
^requires a reviewer to examine the reasonableness of an institution or programs case 
that it is meeting its goals for student learning in the aggregate, given the evidence 
submitted. This might imply examining the adequacy of the evidence itself in scope 
and amount, the reliability and validity of the methods used to generate evidence, 
or the plausibility of the analyses used to make sense of this evidence. As is typically 
the case for regional accreditors, moreover, it might suggest that particular attention 
be given to examining whether or how the institution or program actually uses the 
resulting data for internal improvement. 

Although accrediting organizations ought to be concerned about both indi- 
vidual student attainment and overall instructional effectiveness, the “center of 
gravity” that each accreditor adopts on this continuum is important because many 
of the practical details of its approach to review will automatically follow. For 
example, the more the choice tilts toward assuring the competency of individual 
graduates, the more the focus of review must center on what every graduate knows 
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and can do and on the institution s or program s own internal system of quality 
assurance. The more the choice moves toward overall instructional goal attainment, 
the more the focus of review will be placed on the adequacy of the research designs 
and instruments that the institution or program employs to gather evidence of its 
overall effectiveness and how it has used this evidence for improvement. Once 
again, however, both will require direct evidence of student learning outcomes if 
the issue is to be addressed effectively. 

■ Focus of Review. This dimension addresses the extent to which the review 
actually does examine direct evidence of student achievement. At one end of 
this continuum, direct evidence is invariably (and perhaps exclusively) required. 
Depending upon the choice made on the dimension above, however, such evi- 
dence may or may not be gathered for all students. If the goal is to determine 
overall program effectiveness, for example, only representative samples might 
be examined. And the evidence presented might take many forms, ranging 
from the results of standardized examinations, faculty-designed comprehensive 
or capstone examinations, authentic performance assessments, deliberately 
constructed portfolios, and samples of student work generated in response to 
typical faculty course assignments. But the process must include the presenta- 
tion of direct evidence of student learning by the institution or program and 
its critical evaluation as a prominent part of the review. 

At the other end of this continuum, the focus of review is placed exclusively 
on the adequacy of the institution or programs processes for assuring adequate 
levels of student attainment. Typically, this would include such matters as curric- 
ulum review and approval, policies for student progress and good standing, poli- 
cies governing the award of a degree, grading policies and standards, and processes 
for reviewing and improving faculty teaching performance. Actual student work 
or the results of student assessments of learning might not be examined at all 
in such a review. The process of academic audit in which established protocols 
and audit trail methods are used to determine a) the overall quality of the pro- 
cesses in place compared to a standard “ideal type” for such processes, and b) 
the extent to which the institution actually follows its own processes is at this 
end of the continuum. So is the more traditional and familiar concept of aca- 
demic program review. 

While most accreditors will likely wish to pay attention to both direct evi- 
dence of student learning and internal quality assurance processes, the center of 
gravity of the choice they make on this dimension may again have substantial 
practical consequences. At present, for example, most regional accrediting organ- 
izations call for some direct evidence of student academic achievement, while 
specialized accreditors are mixed in this respect. But most accreditors lack experi- 
ence in evaluating the adequacy of this evidence and almost none have established 
clear standards against which to do so. Partly as a result, the discussion often 
focuses excessively on methodology instead of looking at actual levels of student 
attainment. Equally often, the kinds of evidence of student learning actually 
collected rarely go beyond indirect measures like self-reported attainment on 
surveys or focus-group interviews. Similarly, current accreditation practices 
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frequently examine the policies that institutions and programs establish to assure 
quality, but they rarely do so with the structure, depth, and methodological 
sophistication typical of the academic audit. Consequently, their reviews do not 
usually address how effectively the institution or program is actually carrying out 
its own processes or the extent to which these processes determine its ability to 
consistently deliver graduates who meet established standards of attainment. In 
sum, the choice that accreditors make on this dimension, consciously or uncon- 
sciously, has a decisive impact on the degree to which they are actually addressing 
the issue of student learning outcomes. 

Positions on each of these three dimensions of choice thus define a range of 
quite different postures. But although the choices made on each dimension are in 
principal independent, the actual choices made tend to define a relatively limited set 
of profiles. Some of these, moreover, are of only limited value in effectively address- 
ing the issue of student learning outcomes unless they are undertaken in a particular 
way. Among the most prominent of these modal types are the following: 

■ Program Assessment. This approach is the most typical of current accreditation 
practice, and it is the result of a specific combination of policy choices: 
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Summary of Decisions About Evidence 
and About the Use of Evidence of Student 
Learning Outcomes in Accreditation Review 
and in the Determination of Quality 

Accreditors need to address six core questions in order to effectively incorporate student learning 
outcomes into accreditation review and in the determination of institutional and programmatic 
quality: 

1, What is a “student learning outcome?” 

An “outcome” is something that happens to an individual student as a result of his or her 
attendance at a higher education institution or participation in a particular course of study. 

But there are many types of outcomes other than student learning. A “student learning outcome , ” 
in contrast, is properly defined in terms of the particular levels of knowledge, skills, and abilities 
that a student has attained at the end (or as a result) of his or her engagement in a particular 
set of collegiate experiences. 

2, What counts as evidence of student learning? 

Evidence of student learning can take many forms, but must involve a direct examination 
of student levels of attainment — either for individual students or for representative samples 
of students. Examples of the types of evidence that might be used include (but are not 
limited to): 

■ faculty-designed comprehensive or capstone examinations and assignments 

■ performance on external or licensure examinations 

■ authentic performances or demonstrations 

■ portfolios of student work over time 

■ samples of representative student work generated in response to typical course assignments 

Evidence such as survey self-reports about learning, focus groups, interviews, and student 
satisfaction studies are certainly useful in the accreditation process, but do not constitute direct 
evidence of student learning outcomes. 

3, At what level (or for what unit of analysis) should evidence of student learning 
outcomes be sought? 

Different accreditors may choose different levels of aggregation when seeking evidence 
of student learning outcomes, depending upon their purposes. Options include: 

■ individual student 

■ specified groups or aggregations of students 

■ courses or groups of courses 

■ programs or schools within an institution 
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■ institutions 

■ combinations of the above 

4. To what extent should particular student learning outcomes be specified by accreditors? 

This choice must be made explicitly by individual accreditors, depending upon their 
circumstances. Options range from: 

■ complete prescription of outcomes by accreditor (for example, specific professional skills 
required for practice) 

■ accreditor expects institution (or program) to choose and define outcomes 

■ both (for example, core set of outcomes on which accreditor and institution agree) 

5. What models are available to accreditors when choosing an approach? 

Different combinations of policy choices determine the approach to addressing student learning 
outcomes that any particular accreditor develops. Among the most prevalent and useful points 
of departure are: 

■ program assessment in which outcomes are determined largely by the institution or program, 
the effectiveness of program or institution as a whole is the focus of interest, and direct 
evidence of student learning is collected. [Note: care should be taken here that there really 

is direct evidence of student learning available.] 

■ academic audit in which the effectiveness of student learning outcomes is examined indi- 
rectly by looking at the adequacy of institutional (or program) processes for assuring quality. 
[Note: useful for determining the effectiveness of quality practices, but does not involve 
direct evidence of student learning outcomes.] 

■ academic standards audit in which the adequacy of academic standards for grading and award- 
ing credit is checked by direct reference to actual assignments, requirements, and student work. 
[Note: follows the audit methodology above, but incorporates direct evidence of student 
academic achievement into the audit process.] 

■ third-party certification in which student competency is examined directly by an external 
body. [Note: not typically undertaken by accreditors, but useful to the accreditation process 
as a solid form of evidence.] 

All are useful to accreditation, so long as care is taken to incorporate direct evidence of student 
learning. 

6. What issues should be anticipated? 

A number of important issues arise when any accreditor addresses the topic of student learning 
outcomes as part of the accreditation process. Among them are: 

■ what standards of evidence will be used? 

■ how will evidence be used in determining quality (and in making an accreditation decision)? 

■ how will faculty be involved? 

■ how will the interests and concerns of external stakeholders be addressed? 
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approach encourages the development and deployment of special-purpose assessment 
techniques like portfolios, examinations, and surveys that are administered in addi- 
tion to regular faculty grades. But unless care is taken to ensure that institutions 
understand that direct evidence of student attainment is required, this approach can 
also easily slip into the typical “program review,” in which only processes are generally 
examined. 

Strengths of this approach, when done properly, are that it directly encourages 
institutions and programs to examine student learning in a visible and actionable 
fashion, and that it provides direct and credible evidence about the overall extent to 
which intended learning outcomes are being achieved. Drawbacks are that it may 
encourage institutions and programs to develop a duplicative assessment superstruc- 
ture” that is largely directed toward external reporting, rather than inducing faculty 
to take day-to-day collective responsibility for learning, and that it does not guarantee 
that all graduates of the institution or program have met established goals for learn- 
ing. When adopting this approach, moreover, accrediting organizations need to pay 
particular attention to the extent to which the results of assessment are actually being 
used, and the kinds of changes in instruction that are made in response. 

■ Academic Audit. While the principal examples of program assessment can be found 
among U.S. regional accrediting organizations, the prototype academic audit systems 
were developed in the early 1990s in the United Kingdom, New Zealand, and Hong 
Kong. In contrast to program assessment, audits main focus is not learning outcomes 
perse, but rather the adequacy of the processes that the institution employs to assure 
the academic integrity of its credentials. Accordingly, this approach is equally the 
result of a distinctive set of policy choices on the three dimensions discussed: 
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to decide. On the second, the effectiveness of the institution or program as a whole 
remains the primary object of interest — though the processes that are examined in 
audit are supposed to be those that ensure the integrity of its credentials. On the 
third dimension, however, no direct evidence of student attainment is typically 
presented, though the institutions “assessment system” may be included among the 
processes under review. (In saying this, though, it is important to remember that all 
of the countries that have adopted audit also have some kind of external examiner 
system that simultaneously looks at direct evidence of student attainment.) 

Proponents of this approach claim that it can avoid some of the major draw- 
backs of program assessment because it does not require the development of 
additional — and often controversial — direct measures of learning outcomes. The 
fact that audit is relatively unobtrusive, that it relies on documentation already 
in place at the institution rather than demanding additional “self-study,” and that 
it employs well-developed review protocols to examine real processes in detail 
during the site visit, are also claimed as strengths of the process. But classic aca- 
demic audit has major weaknesses as well — especially in the current U.S. context. 
The foremost is the fact that, by itself, the approach does not require the institution 
or program to produce any direct evidence of student learning outcomes. At the 
same time, because it is principally focused on “inspecting” processes already in 
place, it does not explicitly encourage systematic improvement. 

■ Auditing Academic Standards. One virtue of the framework for choicemaking 
posed by the three dimensions, though, is that it can suggest new kinds of models 
for review. One promising example is the notion of “auditing” academic standards 
instead of internal processes. This approach involves using the kinds of audit mecha- 
nisms employed in Europe and Australia, but directing them explicitly to examine 
the ways in which an institution or program assures the attainment of student 
learning outcomes: 



Chart 2C 

Auditing Academic Standards 





CREDITATION AND STUDENT LEARNING OUTCOMES: 



A PROPOSED POINT OF DEPARTURE 

24 



Page 17 



On the first dimension, the specification of outcomes remains predominantly the 
province of each institution or program. On the second, though, the focus of atten- 
tion shifts to the individual student, because the primary objective is to determine 
the degree to which the institution or program can credibly claim that all of its 
graduates meet acceptable standards of attainment through the academic standards 
that it upholds in its grading processes and in the award of its credentials. On the 
third dimension, direct evidence of student achievement is clearly required. Unlike 
the program assessment model, however, such evidence would most likely be pre- 
sented (and “audited”) in the form of actual student work products, generated 
naturally in the course of a student s academic career. 

In practice, this approach most resembles the “external examiner” system in 
the U.K. Under this system, representative samples of actual student work are read 
by third-party examiners on a periodic basis to ensure that local grading standards 
are consistent with established academic expectations. The approach to accreditation 
currently being developed by the Teacher Education Accreditation Council (TEAC) 
also has much in common with this approach, although the primary focus here is the 
program s assessment system. Potential strengths of this approach are its clear focus 
on student academic achievement and its authentic quality. Potential weaknesses are 
the same as any audit — the fact that the process does not in itself induce improve- 
ment and that there is no guarantee that all students meet all standards. 

■ Third-Party Certification, Although not likely to be employed by accrediting org- 
anizations, the notion of third-party certification serves as a final illustration of how 
the proposed framework for choice- making can define a wide range of approaches. 
Essentially, certification demands direct demonstrations of competence on the part 
of each graduate in order to ensure that specified levels of mastery are being attained. 
This approach is familiar and is widely practiced by professional licensing bodies 
through some kind of examination system. Despite its radical differences from extant 
accreditation approaches, this approach also follows naturally from a particular set of 
policy choices on the three dimensions: 



Chart 2D 

Third-Party Certification 



Complete 

Institutional 

Discretion 



Competency 
Attainment for + 
Individuals 



Processes 
for Quality < 
Assurance 




► 






> 



Outcomes 
Dictated by 
Accreditor 



Overall Program/ 
Institutional 
Effectiveness 



Direct Evidence 
of Student 
Achievement 



ERIC 



ge 18 



COUNCIL FOR HIGHER EDUCATION ACCREDITATION 

25 



On the first dimension, intended student learning outcomes are clearly specified 
and, indeed, provide direct guidance for constructing the needed certification assess- 
ments. On the second, the individual student is the main, if not the exclusive, focus 
of attention. And on the third, direct evidence of student achievement is the sole 
basis on which certification is granted. 

The resulting system is extremely credible and has the virtue of directly 
supporting the program or institutions “guarantee” that the credentials of its 
graduates are valid. On the other hand, the lack of flexibility inherent in this 
approach and its exclusive focus on individual student attainment makes it largely 
inappropriate for accreditation purposes — though an institution or programs 
participation in, and results obtained by its students through, such certifications 
can be extremely valuable evidence in any accreditation process. 

These four illustrations of how the three dimensions of choice can be applied 
are not intended to be exhaustive. Rather, they are advanced to demonstrate the 
flexibility of the scheme in evolving quite different ways to ground a policy stance 
about evidence of student learning outcomes. By making conscious and deliberate 
choices about how to proceed on each of these three dimensions, different actors may 
legitimately end up with very different profiles. But by approaching the task in the 
context of a visible common framework, their distinctive choices can be more widely 
understood and related within the accrediting community. At least as important, the 
differences among them can be more clearly and persuasively explained to those 
outside it. 

Some Enduring Issues 

As accreditors approach the task of developing an appropriate response to the challenge 
posed by student learning outcomes, a number of issues repeatedly arise. Some are 
operational, such as whether and how to include particular kinds of stakeholders or 
how to ensure appropriate levels of faculty involvement. Others are conceptual, such 
as what kinds of evidence should be considered (and how the quality of presented 
evidence ought to be judged), or what to emphasize in a review. In each case, there is 
no single “right” answer about how to proceed. But clearly identifying the nature of 
the issue, and noting the range of choices that might be made to address it, is of value 
in itself. 

■ Standards of Evidence* This issue concerns the kinds of evidence of student learn- 
ing that should be considered acceptable by an accreditor, whatever the approach 
adopted. To some extent, of course, answers will depend on the choice that the 
agency has already made about direct evidence of student attainment. But once it 
is decided that some kind of evidence will be needed, accreditors will need to define 
what “good” evidence looks like. One set of standards here might address the design 
of an institution or programs assessment system, focusing specifically on the extent 
to which this design is capable of credibly demonstrating that established goals for 
learning are being attained. Specific elements of design that might be established as 
standards here could include: a) comprehensiveness , or the degree to which the assess- 
ment system is capable of providing evidence about the full range of student learning 
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outcomes established by the institution or program; b) multiple judgment, or the 
extent to which multiple sources of evidence are used in a mutually reinforcing way 
to examine outcomes; c) multiple dimensions, or the degree to which different facets 
of student performance with respect to established learning outcomes can be investi- 
gated so that patterns of strength and weakness can be identified (and addressed); 
and d) direct evidence, or the extent to which the approach relies upon direct mea- 
sures of student attainment instead of self-reports about learning or proxy indicators 
of attainment like graduation rates or graduate placement. 

Another set of standards might address the quality of what is presented against 
recognized attributes of credible evidence. Standards here might address such things 
as: a) validity, or the degree to which the methods used yield evidence that adequately 
reflects the nature of the underlying ability being assessed; b) reliability, or the extent 
to which these methods produce the same result consistendy over time or across 
contexts; and c) representativeness, or the degree to which the evidence presented — 
especially if based on a sample — truly represents the performance of the wider pop- 
ulation. Standards like these, of course, are drawn directly from the measurement 
literature and can be applied with varying degrees of precision. Some regional 
accreditors, for example, have addressed this question in general terms in their 
publications. Others, like TEAC, have developed explicit review protocols based 
on the educational measurement literature. 

Like any of the dimensions of choice outlined earlier, where a particular accredi- 
tor chooses to come down on the question of evidential standards might legitimately 
differ. But as external stakeholders focus increasingly on student learning outcomes, 
it is clear that some kind of standards to address evidential quality will be needed. 

■ Value-Added. This issue centers on how (and whether) to require institutions or 
programs to assess the extent of student growth and development with respect to 
learning in addition to simply the “adequacy” of current attainment levels. For an 
accreditor, the choice here will depend largely on the purposes of review and, in some 
cases, attention to both may be appropriate. For specialized accreditors, choices on 
this dimension will tend toward end-point assessment — largely because of the over- 
whelming importance of assuring that all program graduates have attained the levels 
of competence needed for effective professional practice. For institutional accreditors, 
choices may tend more toward demonstrating growth and development in order to 
establish the institution or programs unique contribution to learning and to encour- 
age improvement. But these tendencies will be approximate at best. Again, the point 
is that the choice needs to be deliberate and defensible, whatever the resulting 
approach may look like. 

Accreditors also need to be aware of the substantial methodological issues that 
accompany any attempt to assess “value added.” First, determining growth requires 
information about students’ knowledge and skill at the point at which they entered 
the program in addition to evidence of ultimate attainment — both of which may 
contain errors. Second, claiming credit for development requires an attribution of 
cause — that the instruction received was really responsible for any observed change 
in student ability levels. When looking at evidence of “growth,” therefore, reviewers 
will need to be mindful of both these issues. 
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■ Use of Results. This issue focuses on how the institution or program uses the 
evidence it gathers about student learning outcomes to improve curriculum and 
pedagogy. All accrediting organizations will claim that they are interested in “im- 
provement.” But the extent to which they consciously direct their standards and 
review processes toward examining institutional improvement efforts may vary 
considerably. Some accreditors may accept fairly crude 
and indirect forms of evidence if it can be established that 
the institution or program is learning something of value 
and is acting on what it knows. Others will rightly insist 
on much higher evidential standards because their primary 
purpose is to validate the institution or programs claims 
that its graduates meet specific levels of attainment. Re- 
gardless of where an accrediting organization places its 
emphasis on this issue, some way to determine the level 
and quality of an institutions or programs use of results 
will be required. 

Systematic attention to this matter, moreover, might 
involve developing rubrics or standards around several 
aspects of “use.” Among them might be: a) dissemination , 
or the degree to which the results of assessment are com- 
municated to appropriate audiences and the degree to 
which selected members of the institutional or program 
community are aware of them; b) extensivity , or the breadth 
with which such results are known across departments, 
units, subfields, or individual faculty; c) intensivity y or the degree to which results 
of assessment are incorporated into regular discussions and decision-making pro- 
cesses; and d) visible change , or the extent to which specific changes in curriculum, 
pedagogy, or policy can be credibly attributed to the presence of assessment results. 
Audit processes might be usefully employed to examine each of these attributes 
of use, and institutions or programs might be given specific guidance to address 
them when constructing the body evidence that they present. 

■ Reliance on Outcomes. This issue centers on the relative weight that an accreditor 
places on evidence of student attainment when making decisions about the accred- 
ited status of an institution or program. On the one hand, there is clear pressure 

to increase die salience of such evidence in relation to more traditional “markers” 
of quality based on resources and processes. And in the case of distributed and 
distance-delivered instructional designs, most of these traditional markers simply 
do not apply. On the other hand, there remains a substantial consensus across the 
accreditation community that evidence of outcomes alone is insufficient to make 
adequate or credible judgments. In addition to inherent doubts about the validity 
and reliability of assessment, accreditors know that they will encounter resistance 
from faculty and institutions to basing judgments of “quality” solely on evidence 
of student attainment. 

Most accreditors are responding to the implied tension by increasing the 
visibility of standards and review requirements that address student learning, 



Enduring Issues... 

■ Standards of Evidence: What is acceptable 
evidence of student learning? 

■ Value-Added: Should we assess the extent 
of student growth and development? 

■ Use of Results: How will evidence of 
student learning outcomes be used to 
improve curriculum and pedagogy? 

■ Reliance on Outcomes : What is the relative 
weight an accreditor places on evidence 
of student attainment? 

■ Faculty Involvement: Are faculty actively 
involved in the process of developing 
evidence related to student learning? 
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while changing the nature of more traditional “resource/process” requirements to 
increase their flexibility. Examples include reduction in the number and specificity 
of requirements that address faculty resources, instructional activities, and the 
adequacy of student support resources. And given current (and anticipated) DOE 
concern that recognized accreditors address some very specific components of 
educational programs like student complaint procedures, it is unlikely that the 
accreditation community will be able to entirely abandon process standards even 
if there was a desire to do so. 

But the manner in which an accreditor chooses to address student learning 
outcomes remains important. One approach — and so far the prevalent approach 
— is to adopt additional standards and review elements to address student learning 
outcomes. Examples include the “institutional effectiveness” standards adopted 
by a number of regional accreditors and the separate “educational effectiveness” 
review process just adopted by WASC. This approach has the virtue of calling 
explicit attention to matters of student learning in ways that are visible to both 
institutions and stakeholders. But it may have the unintended consequence of 
isolating conversations about learning from the mainstream of discussion and 
encouraging them to be dealt with as “compliance” issues. The alternative is to 
attempt to integrate attention to student learning outcomes throughout the 
agency’s review standards and processes in a way that cuts across all aspects of 
an institution or program. This approach has the strength of emphasizing that 
student learning is the ultimate “bottom line” and that all resources and processes 
ought to be aligned to facilitate it. But because addressing learning outcomes is 
both unfamiliar and hard, it may induce institutions to fall back into the mode 
of simply describing their programs in traditional terms, absent a specific “learn- 
ing standard.” Certainly, there is no single right way out of this dilemma. But 
accreditors should be aware of its existence and should be clear about what choices 
that they have made and why. 

■ Faculty Involvement. This issue centers on the extent to which faculty are 

actively involved in the process of developing evidence related to student learning 
outcomes and, as a consequence, in the accreditation process itself. On the one 
hand, virtually all accrediting organizations will require that faculty be “actively 
and broadly involved” in every aspect of assessment. On the other, the actual 
collection and interpretation of evidence about student attainment — especially if 
it is implemented in the form of the typical “assessment program” visibly distinct 
from day-to-day teaching and learning activities — can be a matter of considerable 
technical complexity. As a result, line faculty may or may not be directly involved 
in every aspect. 

Given this situation, accreditors need to think carefully about what they really 
mean when they require “faculty involvement.” At one level, faculty should cer- 
tainly be broadly and actively involved in establishing goals for learning in the first 
place. As a result, standards and review processes should indeed examine the extent 
of faculty involvement in goal-setting, and whether it has consequences for curricu- 
lum/course design and for teaching. At some institutions, for instance, “goals” for 
general education have been established largely for the purposes of designing 
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assessments, not for the purposes of guiding pedagogy and instructional design. At 
a second level, faculty need not always be directly involved in the process of select- 
ing or designing methods to assess student attainments. If such methods are used to 
directly certify individual student performance, they will and should be involved. 
But if they are used only periodically to determine overall program effectiveness, 
not every faculty member need be consulted and even fewer will likely do the work. 
What is important is that those who make the decisions and who collect the 
requisite evidence are accorded the legitimacy to do so by their peers. At a third 
level, faculty ought to be visibly involved in interpreting and drawing conclusions 
based on the resulting evidence. Again — and especially if a primary intent of 
accreditation is to stimulate local improvement — both standards and review pro- 
cesses should be tailored to determine the degree to which this is happening. At a 
final level, there might or might not be a need for broad faculty involvement in the 
process of assembling evidence about the attainment of student learning goals for 
purposes of accreditation. Under a traditional “self-study* * approach in which an 
institutional presentation is carefully crafted to make a case for educational effec- 
tiveness, a broad level of involvement should probably be expected. In the case of 
an academic audit, though, only a few individuals — and likely administrators — 
may be involved in the process of assembling evidence — though auditors should 
be encouraged to inquire into the role faculty actually play in the processes being 
audited. Again, the point is not simply to call for “involvement,” but to carefully 
think through what “involvement” really means. 

■ Stakeholder Involvement. This issue concerns the extent to which learning out- 
comes should be defined and assessed strictly from the perspective of the institution 
or programs own faculty, or whether external voices — like those of employers or 
relevant professional communities — ought to be heard. For specialized accreditors, 
the choice is clear: professional communities should be visibly and frequently in- 
volved in at least the process of establishing student learning outcomes. Alternatively, 
the outcomes established by the programs faculty should be demonstrably aligned 
with accepted professional standards of practice or acknowledged areas of compe- 
tence. More specifically, the kinds of evidence of student attainment generally 
accepted should at minimum include such things as pass rates on relevant profes- 
sional licensing examinations and tracking performance in later professional practice. 
For institutional accreditors the choice is less clear and will depend largely on insti- 
tutional mission. To the extent that the institution offers vocational or professional 
programs where a clear external community of practice is present, stakeholder 
involvement in establishing goals for student learning should be prominent. For 
both, it is important to note that the DOE also has a position on this issue: if a 
program prepares students for practice, accreditors are obliged to ensure that 
appropriate stakeholder groups are involved in assessment. 

For all of these issues, choices will need to be made about how to proceed. But 
by understanding the underlying dimensions of choice more fully, accreditors will be 
in a better position to both make informed decisions and to explain what they have 
done to the many audiences who may be interested. 
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Concluding Thoughts 

Two things are clear from the previous discussion. First, the task of addressing student 
learning outcomes has no single or simple answers. Second, it is imperative that accre- 
ditors take it on. What is required for them to do so? Three lines of collective action 
seem warranted. 

■ First, the accrediting community needs to develop a coherent and understandable 
way to explain its collective approach to the matter of evidence of student learning 
outcomes to outside stakeholders. This rationale must make it clear that there are 
appropriate differences among accreditors in how they choose to engage student 
learning, but that all are doing so in appropriate and rigorous ways. But it must also 
make clear that they are requiring some measure of direct student attainment — not 
merely proxies for it — in their various approaches. 

■ Second, accreditors need a language with which to talk to one another about what 
they are doing. This may or may not require the development of a common vo- 
cabulary. But it does require a common conceptual framework that allows varied 
members of the community to understand the key distinctions and similarities 
among their approaches. 

■ Third, all would benefit from the development of common resources . These might 
include sharing review approaches and techniques like standards of evidence or ways 
to “audit” institutional or program assessment efforts. It might also include more 
fundamental research and development efforts directed toward creating better tools 
for examining learning or identifying best practices. There is a growing fund of 
experience to build on here, both within and outside the accreditation community. 
But the growing body of experience has not yet been effectively “rounded up” for 
use by the community in common. 




COUNCIL FOR HIGHER EDUCATION ACCREDITATION 



2001-2002 CHEA Board of Directors 



John T. Casteen III, Chair y President, University of Virginia 
William DeLauder, Vice Chair , President, Delaware State University 
Eleanor Baum, Secretary , Dean, Engineering School, The Cooper Union 
Edward Donley, Treasurer , Former Chairman, Air Products and Chemicals, Inc. 

Gordon A. Haaland, Immediate Past Chair y President, Gettysburg College 

Vernon O. Crawley, President, Moraine Valley Community College 

Alfredo G. de los Santos Jr., Research Professor, Arizona State University 

S. Malcolm Gillis, President, William Marsh Rice University 

Robert B. Glidden, President, Ohio University 

Ira Lechner, Former Trustee, Randolph Macon College 

Karen W. Morse, President, Western Washington University 

Charles R. Nash, Vice Chancellor for Academic Affairs, The University of Alabama System 
Lloyd E. Reuss, Former President, General Motors Corporation 
Piedad F. Robertson, Superintendent/President, Santa Monica College 
Arthur J. Rothkopf, President, Lafayette College 

How to Contact CHEA 

CHEA is pleased to provide information and assistance related to accreditation issues 
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